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Executive Summary 

California’s new accountability 
system originated in the radical 
decentralization of power and au- 
thority from Sacramento to local 
schools and their communities 
brought about by the Legisla- 
ture’s adoption of the Local Con- 
trol Funding Formula (LCFF) in 
2013. Under California’s previ- 
ous accountability policies and the 
federal “No Child Left Behind” 
law, the state set performance 
targets for schools and districts 
based almost entirely on students’ 
standardized test scores. Schools 
that fell short of their targets were 
subject to a variety of increasingly 
harsh sanctions, ranging from des- 
ignation as a “failing” school to re- 
constitution or closure. 

California’s new accountability 
system is different from the pre- 
vious system in nearly every im- 
portant respect. The new system 
is grounded in the concept of re- 
ciprocal accountability: that is, 

every actor in the system — from 
the Capitol to the classroom — 
must be responsible for the aspects 
of educational quality and perfor- 
mance that it controls. 

Key Elements of California’s 
New Accountability System 

The state has made three funda- 
mental commitments: 


• To pursue meaningful learn- 
ing for students - through the 
adoption of new standards and 
curriculum frameworks more 
focused on higher order think- 
ing and performance abilities; 

• To give schools and districts 
the resources and flexibility they 
need to serve their communities 
effectively - through the new 
LCFF, which allocates funds 
based on student needs and al- 
lows communities to determine 
where the funds should be spent 
to achieve the best results; 

• To provide professional learning 
and supports for teachers and 
administrators - through stron- 
ger preparation and ongoing 
professional development. 

At the same time, the state has 
adopted three complementary 
mechanisms to hold schools and 
districts accountable: 

• Political accountability, op- 
erationalized through Local 
Control Accountability Plans 
(LCAPs), created by districts 
with their communities, updat- 
ed annually, and reviewed by 
county agencies. The LCAPs, 
intended to ensure that re- 
sources are used wisely and ef- 
fectively, articulate local goals 
for schooling and report out- 
comes. 
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• Professional accountabil- 

ity, through effective licensure, 
professional development, 
and productive evaluation, to 
ensure that educators deliver 
high-quality instructional and 
other services to their students, 
and 

• Performance accountability, to 
ensure continuous improve- 
ment in the performance of 
schools across the state’s eight 
priority areas, plus other priori- 
ties identified locally. The eight 
priority areas include student 
achievement, student engage- 
ment, school climate, parent 
involvement, provision of basic 
services, curriculum access, and 
implementation of the state’s 
new standards. 

Creating a Continuously 
Improving System 

With the simultaneous imple- 
mentation of LCFF and LCAP, 
along with new standards, cur- 
riculum, and assessments, schools 
and school districts throughout 
California have a unique opportu- 
nity to reconfigure themselves as 
learning organizations, commit- 
ted to continuous improvement 
and explicitly organized to sup- 
port experimentation, evaluation, 
and organizational learning. Key 
features of a continuously improv- 


ing education system include: 

• Learning supports (materials 
and professional development) 
for the continuous improvement 
of curriculum, teaching, assess- 
ment, and student support strat- 
egies; 

• Information systems for keeping 
track of what schools and districts 
are doing and to what effect; 

• Ongoing review of school and 
district efforts and outcomes, 
including self-assessment and re- 
view by experts and peers; 

• Thoughtful innovation and 
evaluation to support teachers, 
schools, and school districts as 
they experiment with promis- 
ing policies and practices in ways 
that are a) informed by existing 
knowledge about those practices, 

b) designed to support serious 
evaluation of their implementa- 
tion challenges and effects, and 

c) intended to support broader 
adoption of successful approach- 
es and abandonment of unsuc- 
cessful ones; and 

• Knowledge dissemination strat- 
egies (through a central reposi- 
tory of research and exemplars, 
convenings, networks, and lever- 
aged supports) so that successful 
practices become widely known 
and supported in their wider 


adoption / adaptation. 

In this report we describe what it 
will take for California to create 
a continuously improving educa- 
tion system. We focus especially 
on the critical role that the new 
California Collaborative for Edu- 
cational Excellence (CCEE) will 
play in the new system, and offer 
design principles to inform the 
decisions that will guide its orga- 
nization and operation. 

Two key pillars are needed to sup- 
port continuous improvement 
in California’s education system. 
The first is an information and 
reporting system that can enable 
educators and state agencies to as- 
sess how things are working and 
how well students are learning. 
The second is an agency - the new 
CCEE - that can create supports 
for learning, knowledge sharing, 
and evaluation, as well as for direct 
intervention and improvement in 
the work of schools and districts 
that are struggling to provide an 
adequate education. These two 
pillars are linked, as the long-term 
effectiveness of the CCEE de- 
pends in substantial measure on a 
high-quality information system. 

Organizing and Reporting In- 
formation 

One key to a sustained process of 
continuous improvement is the 
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regular review of data to guide 
diagnosis of local strengths and 
weaknesses and to identify strate- 
gies and practices to support im- 
proved performance. In the state’s 
new accountability system districts 
and schools must he able to access 
and use data on state and local pri- 
orities easily, and County Offices 
of Education (COEs) and Charter 
School Authorizers (CSAs) must \ 
have the capacity to evaluate local 
performance in a reliable, consis- 
tent manner. To meet these needs, 
and to guide effective assistance for 
struggling schools, the state must 
have school performance data that 
can be compared across schools 
and districts and aggregated on a 
statewide basis. 

Rather than maintain competing 
reporting systems based on exist- 
ing measures, California should 
develop a coherent approach that 
places the state’s eight priorities at 
the heart of a unihed accountabil- 
ity system, augmented with local 
measures that reflect additional 
community goals and priorities. 
The state should replace the Aca- 
demic Performance Index (API), 
the State Accountability Report 
Card (SARC), and the current on- 
line reporting system with a dash- 
board of measures that reports 
progress on the state’s priorities. 

Instead of seeking to rank schools 


and districts on a single measure, 
the dashboard will reveal how they 
are doing in relation to criteria for 
performance and how they are im- 
proving in different areas. The use 
of multiple measures is much more 
informative than a single index for 
planning and improvement efforts. 
Eike the dashboard on a car - which 
provides indicators of speed, dis- 
tance traveled, fuel, fluids, tire pres- 
sure, and more — the combination 
of measures provides information 
about where to look further in order 
to figure out how things are work- 
ing and what may need attention. 

An additional policy lever as the 
state moves toward a more aligned 
accountability system is the evalu- 
ation rubric that will be used in 
examining and assessing ECAPs to 
determine when a school or dis- 
trict needs assistance. An on-line 
statewide reporting tool to support 
ECAPs could incorporate the ru- 
bric’s standards and could provide 
available state data for most of the 
ECAP indicators. The dashboard 
could be supplemented by a set of 
student, teacher, and parent surveys 
that include some common ques- 
tions across the state in addition to 
locally identified questions. The 
surveys could provide information 
about many of the indicators, such 
as school climate and services, with- 
out creating burdensome reporting 


requirements for schools and dis- 
tricts. 

Eocal schools and districts could 
draw from this tool and add their 
own indicators and data for prior- 
ity areas where performance data 
are not readily compared, or for 
areas where local goals have been 
included in the ECAP. An ex- 
ample of what such a tool could 
look like, modeled on the Alberta 
(Canada) Results Report Card, is 
included in Appendix B of this re- 
port. 

The primary use of the ECAP eval- 
uation rubrics will be to provide 
guidelines against which schools 
and districts can assess their own 
progress, in order to guide ongo- 
ing improvements. Some, how- 
ever, will need additional help to 
be successful. The first line of 
technical assistance will be Coun- 
ty Offices of Education, while the 
ultimate responsibility will rest 
with the CCEE. 

The California Collaborative 
for Educational Excellence 
(CCEE) 

The cornerstone of California’s 
new accountability system is the 
California Collaborative for Edu- 
cational Excellence (CCEE), not 
because the CCEE can or will 
do most of the work required to 
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support continuous improve- 
ment in the system but because 
it is the agency that is ultimately 
responsible for making the system 
work. Under the LCFF statute 
the CCEE’s main responsibility 
is to help develop and implement 
strategies to improve local perfor- 
mance in schools and school dis- 
tricts where action by the COE or 
CSA has failed to launch a cycle of 
continuous improvement. This 
entails two main tasks. 

Eirst, the CCEE must provide di- 
rect assistance to schools, school 
districts, COEs and CSAs that 
are falling short of their goals and 
obligations, including both those 
that are identified by the SPI and 
those that request assistance on 
their own. To fulfill this respon- 
sibility the CCEE will have to 
conduct skillful, diagnostic in- 
vestigations for schools that have 
requested or have been identified 
for assistance. These investiga- 
tions should be based on a School 
Quality Review process that en- 
gages experts and peers in school 
visits and consultation with lo- 
cal stakeholders. The resulting 
diagnostic reports should iden- 
tify weaknesses and problems and 
suggest context-sensitive strategies 
for remediation, with the goal of 
strengthening local capacity for 
organizational learning and con- 


tinuous improvement. 

Second, the CCEE must organize 
assistance for improvement. This 
could take several forms, including, 
as in some other states, the training 
and deployment of a cadre of Distin- 
guished Educators — accomplished 
teachers, principals, and superinten- 
dents — who are intensively prepared 
and made available to work closely 
with schools and districts that are 
engaged in improvement or turn- 
around efforts. Another promising 
strategy is the creation of school or 
district pairings and networks, con- 
necting schools that are struggling 
to more successful schools that face 
similar challenges. This approach to 
continuous improvement has been 
pioneered in Shanghai, China and 
in the CORE districts in Califor- 
nia. Networking of small groups of 
schools, combined with knowledge 
dissemination strategies, has also 
been used successfully to support 
improvement in other contexts, in- 
cluding England and Ontario. 

In addition to these two core func- 
tions, the CCEE should work with 
others in the state - CDE, COEs, 
and other providers — to support 
knowledge sharing and dissemi- 
nation. In order to support con- 
tinuous improvement throughout 
the education system, and not just 
in individual schools and districts, 
California will need to build its 


capacity to compile and evalu- 
ate information about practices, 
tools, and resources that show 
promise, and to share this infor- 
mation with COEs, CSAs, dis- 
tricts, and schools. 

In addition, in a state that has 
eliminated most of the infrastruc- 
ture for professional learning, the 
CCEE will want to figure out 
what kind of learning supports 
it can contribute to the range of 
learning needs schools and dis- 
tricts will present. One especially 
urgent task is to provide guidance 
to schools and school districts 
about the quality and alignment 
of instructional materials and pro- 
fessional development programs 
that claim to advance the goals of 
new standards implementation. 
Other supports aimed at build- 
ing local capacity could include 
assistance to districts in develop- 
ing professional learning com- 
munities and stronger evaluation 
systems, including Peer Assistance 
and Review strategies. The CCEE 
should help to determine how 
these functions can best be man- 
aged in the state and how they can 
be phased in over time. 

Three fundamental principles 
should guide decisions about or- 
ganizational design of this new 
agency. Eirst, the CCEE should 
employ a core professional staff 
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to review intervention strategies 
and oversee technical assistance 
activities, while contracting 
with multiple partners to pro- 
vide direct assistance to school 
districts and schools. Second, 
the CCEE should engage in 
partnerships that huild on exist- 
ing public infrastructure to the 
greatest possible extent. Finally, 
the design should be scalable, 
to enable the CCEE to respond 
to new expectations and grow- 
ing demand for assistance over 
time with a thoughtful phase-in 
process. This kind of unified 
long-term strategy could en- 
able California to move success- 
fully from a compliance-driven 
system to one that is capable of 
system learning and continuous 
improvement. 
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Supporting Continuous Improvement in California’s Education System 


Introduction 

C alifornians are just begin- 
ning to recognize the scale 
and impact of recent policy 
changes in the state’s education sys- 
tem. The simultaneous adoption 
and implementation of Common 
Core State Standards (CCSS) and 
the Local Control Funding Formula 
(LCFF) promise large and potential- 
ly transformational improvements 
in the state’s schools and classrooms. 
Ensuring that these reforms live up 
to their promise will require educa- 
tors and policymakers to develop 
and pursue an integrated, long-term 
strategy that supports system-wide 
learning about the effects of poli- 
cies and practices and continuous 
improvement at all levels of Califor- 
nia’s education system. 

The engine that will drive this trans- 
formation is California’s new ac- 
countability system, which differs 
from the previous system in nearly 
every important respect. Some key 
parts of the new system are still 
under construction, but the broad 
outline of what it will look like is 
already clear. 

The new system is grounded in 
the concept of reciprocal account- 
ability: that is, every actor in the 
system must be responsible for the 
aspects of educational quality that 


it controls. On the one hand, the 
state has made three fundamental 
commitments: to pursue meaning- 
ful learning for all students, to give 
schools and districts the resources 
and flexibility they need to serve 
their communities effectively, and 
to provide professional learning and 
support for teachers and adminis- 
trators at all levels of the system. As 
shown in Figure 1, these constitute 
three pillars of a new accountability 
system that is designed to support 
continuous improvement: 

• Accountability for meaningful 
learning 

• Accountability for adequate and 
intelligent resource allocation 

• Accountability for professional 
competence and capacity' 


At the same time, the state has ad- 
opted three complementary mecha- 
nisms to hold schools and districts 
accountable: political account- 

ability to ensure that resources are 
used wisely and effectively, pro- 
fessional accountability to ensure 
that educators deliver high-quality 
instructional and other services to 
their students, and performance ac- 
countability to ensure continuous 
improvement in the performance of 
schools and students across multiple 
dimensions. 

In this paper, we first review the 
key elements of California’s new ac- 
countability system, with a focus on 
the reciprocal obligations of state 
and local agencies. We then dis- 
cuss what a continuously improv- 


Figure 1: Key Elements of an Accountability System 
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ing education system would need 
to include, and describe what it will 
take for California to create such a 
system. We conclude with a discus- 
sion of the critical role that the new 
California Collaborative for Educa- 
tional Excellence (CCEE) will play 
in the new system, and offer some 
design principles to inform the de- 
cisions that will guide the organiza- 
tion and operation of the CCEE. 
These design principles are based in 
part on a review of similar initiatives 
in California and in other states and 
countries. Several of these initia- 
tives are described in text boxes 
throughout this paper and in a set 
of appendices. 

California’s New 
Accountability System 

California’s new accountability sys- 
tem originated in the radical decen- 
tralization of power and authority 
from Sacramento to local schools and 
their communities brought about 
by the Eegislature’s adoption of the 
ECEE in 2013. Here we describe 
the new accountability framework 
and the progress of implementation 
to date, flagging considerations and 
issues that will have to be addressed 
as the system takes shape. 

Resource Accountability 

California has pursued resource 
accountability by developing a 
weighted student funding formula 


that allocates funds based on pupil 
needs, assigning a greater weight to 
students living in poverty, English 
learners, and students in foster care. 
As a result, many California school 
districts will receive large infusions 
of new resources under the ECEE 
The law intends to hold local dis- 
tricts accountable for intelligent and 
equitable uses of these funds by col- 
lecting evidence about what districts 
and schools offer their students and 
what the outcomes are, with data 
disaggregated by student race/ eth- 
nicity, poverty, language status, and 
disability status. 

While the ECEE makes the distri- 
bution of resources in California’s 
education system fairer and more 
rational, it does not increase the 
total resources available. Spending 
on education in California remains 
far below the national average, and 
the ratio of adults to students in 
California schools and classrooms 
is smaller than in almost any other 
state. Many instructional programs 
that were slashed during the state’s 
fiscal crisis have yet to rebound. 
California still has far to go in pro- 
viding educators with the resources 
they will need to achieve the goal of 
college and career readiness for all 
the state’s students. 

Local Control and Accountability 
Plans 

Under the ECEE, school districts 


and charter schools are responsible 
for producing Eocal Control Ac- 
countability Plans (ECAPs), which 
lay out local goals and strategies in 
eight priority areas and link them 
clearly and directly to local deci- 
sions about resource use. Educators 
must develop their ECAPs in con- 
sultation with parents, community 
members, and other local stakehold- 
ers. These constituencies are in turn 
expected to hold their schools and 
school districts accountable for set- 
ting the right goals, using resources 
equitably and wisely, and supporting 
steady improvement in educational 
performance for all students. The 
ECAP is expected to link resource 
use to specific strategies for improv- 
ing student achievement and attain- 
ment through the local budget. 

County Offices of Education 
(COEs) and charter school autho- 
rizers (CSAs) are responsible for 
reviewing the ECAPs produced by 
districts and schools under their ju- 
risdiction to ensure that the ECAPs 
comply with state regulations and 
that the allocation of budgetary re- 
sources is consistent with the goals 
that schools and districts set forth. 
These oversight bodies will also 
evaluate school and district progress 
toward the goals that they have set. 
If a local school district is falling 
short of the goals articulated and ap- 
proved in its ECAP, the COE must 
provide guidance and technical as- 
sistance to the district, including the 
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assignment of experts to assist the 
district in turning itself around.^ 

The California Collaborative for 
Educational Excellence 

In cases where COEs lack the re- 
sources or expertise that local dis- 
tricts need to achieve their goals, the 
COE may refer the district to the 
Superintendent of Public Instruc- 
tion (SPI), who may in turn ask the 
California Collaborative for Educa- 
tional Excellence (CCEE) to pro- 
vide technical assistance. Schools 
and school districts may also seek 
assistance from the CCEE on their 
own initiative. The CCEE thus has 
a potentially critical role to play in 
California’s new accountability sys- 
tem as a repository of expertise and 
technical support for schools and 
districts facing challenges that they 
cannot surmount locally. 

As of this writing, however, the 
CCEE does not yet exist. Although 
the statute that created the CCEE 
specified the procedure for appoint- 
ing a five-member governing board 
and appropriated $10 million to 
support initial operations, the board 
has yet to meet, and basic decisions 
about what the CCEE will look like 
and how it will function remain to 
be made. 

Meaningful Learning 

California’s primary strategy to en- 
sure meaningful learning for all 


of the state’s students has been to 
adopt and support implementation 
of the Common Core State Stan- 
dards (CCSS) in English language 
arts and mathematics and the as- 
sociated Smarter Balanced (SBAC) 
assessments. The Next Generation 
Science Standards (NGSS) have also 
been adopted, with new assessments 
promised in several years. Curricu- 
lum frameworks have been or are 
being developed for each of the new 
sets of standards, and the California 
Department of Education (CDE) is 
to bring a broader plan for assess- 
ment to the legislature by 2016. 
All the older tests, except for the 
California High School Exit Exam 
(CAHSEE), are targeted for elimi- 
nation. 

The CCSS and NGSS propose am- 
bitious new expectations for student 
learning and reflect the state’s cen- 
tral goal of increasing the number of 
students who leave school equipped 
with deeper learning skills that in- 
clude critical thinking; communica- 
tion and collaboration skills; creativ- 
ity; and the ability to learn to learn. 

Some of California’s largest school 
districts are part of the Califor- 
nia Office to Reform Education 
(CORE) initiative, through which 
they are working together to build 
curriculum and assessment systems 
intended to support both deeper 
learning and the development of 
social-emotional skills, such as per- 


sonal and social awareness and re- 
sponsibility, collaboration skills, 
resilience, perseverance, and an 
academic growth mindset. These 
are part of the CORE districts’ ac- 
countability approach, which adds 
a social-emotional domain to the 
LCAP requirements. (The CORE 
accountability system is described 
in Appendix A.) 

Moving from a list of standards to 
a curriculum and classroom instruc- 
tion is neither a small nor a simple 
task. The state can provide cru- 
cial support to local educators as 
they navigate this transition. This 
should include supporting the de- 
velopment and distribution of high- 
quality curriculum materials and 
assessment tools and encouraging 
local districts to select and develop 
thoughtful, curriculum-embedded 
assessments of students’ knowledge 
and skills that provide ongoing di- 
agnostic information to support 
learning. Assistance to teachers as 
they assess curriculum frameworks 
and develop local curricula would 
be especially useful, as would a 
more systematic review and vetting 
of textbooks and other instruction- 
al materials to ensure that they are 
high quality and are truly aligned 
to the CCSS. The state should also 
provide support for teacher learning 
and professional development, as 
discussed further below. 
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Professional Competence 
and Capacity-Building 

The implementation of CCSS and 
NGSS dramatically raises expecta- 
tions for California teachers. Under 
the federal No Child Left Behind 
(NCLB) legislation, teachers were 
held strictly accountable for their 
performance on the California Stan- 
dards Tests (CSTs), which measured 
primarily low-level skills.^ One con- 
sequence was that teaching became 
an increasingly scripted activity, 
with little discretionary authority 
for teachers over what to teach or 
how to teach it. Under both CCSS 
and NCSS, in contrast, teachers are 
expected both to engage deeply in 
curriculum design and materials de- 
velopment and to engage their stu- 
dents in deeper learning. For many 
veteran teachers, this represents a 
return to highly valued profession- 
al norms that were steadily eroded 
under NCLB. For some more re- 
cent entrants, however, it represents 
something entirely new, for which 
they at best only weakly prepared. 

To fully implement California’s new 
academic standards, many teachers 
will need to learn pedagogical strat- 
egies that are new to them, and all 
teachers will need to learn how to 
integrate formative assessment into 
their teaching to support continu- 
ous improvement in their own in- 
structional practice. 


This is equally true of principals. 
Under NCLB, principals were sub- 
ject to accountability rules imposed 
on their schools by both state and 
federal authorities, and school lead- 
ership required them to ensure that 
their teachers were faithfully deliv- 
ering standardized material in ways 
that were thought to maximize stu- 
dent performance on standardized 
assessments. As they move forward 
with CCSS and NGSS implemen- 
tation, many California principals 
will need intensive professional de- 
velopment to enable them to pro- 
vide the new kinds of leadership 
that more ambitious goals for teach- 
ing and learning expect and require 
of them. This includes both leader- 
ship inside their schools and deeper 
engagement with parents and other 
interests and constituencies in the 
broader community. 

To address these needs, the Califor- 
nia Commission on Teacher Cre- 
dentialing (CTC) has increased the 
standards for preparation institu- 
tions, requiring them to integrate 
training for the new standards in 
pre-service and induction programs. 
The CTC is also overhauling licen- 
sure and accreditation to create 
greater leverage for program im- 
provement. In addition, the legisla- 
ture allocated $1.25 billion to local 
districts for CCSS implementation, 
some of which was used to support 
professional development. 


Although California is moving to- 
ward fulfilling its commitment to 
guarantee professional competence 
in every school and classroom, much 
more remains to be done. The in- 
frastructure for professional devel- 
opment that once existed (for ex- 
ample, the Subject Matter Projects, 
California School Leadership Acad- 
emy, supports for National Board 
Certification, and direct funding 
of some professional development 
programs) has been almost entirely 
eradicated by budget cuts over the 
last decade or more. 

In addition, the ratio of adults to 
students in schools is lower in Cali- 
fornia than in almost any other state, 
and this has a variety of negative 
consequences. The lack of support 
staff places an inordinate burden on 
classroom teachers, depriving them 
of the time needed to work with 
their colleagues to develop curricula 
and instructional materials or to 
participate in professional develop- 
ment activities. Similarly, princi- 
pals lack the administrative backup 
that would permit them to con- 
duct meaningful evaluation of their 
teachers or to provide their teachers 
with the instructional support that 
they need. Providing teachers and 
principals with the time, knowledge, 
resources, and support that they will 
need to successfully implement the 
CCSS will be a challenge for policy 
makers and educators at all levels of 
California’s education system. 
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Accountability Strategies 
Activated in the New System 

The new approach will rely on three 
accountahility strategies, described 
helow, to ensure that goals are met. 

Political Accountability 

Under the LCFF, the main respon- 
sibility for ensuring that resources 
are used wisely and effectively is 
lodged with local educators and 
their communities as they develop 
and review LCAPs. If these LCAPs 
are not responsive to community 
expectations, or if local educators 
do not pursue the commitments 
articulated in the LCAP, the com- 
munity’s first recourse is politi- 
cal. In local school districts, voters 
can change the composition of the 
school board. In charter schools, 
parents can withdraw their children 
and enroll them elsewhere. For this 
political accountability to operate 
effectively, however, parents and 
other constituencies will need reli- 
able data and analysis that reveal 
whether resources have in fact pro- 
duced the results that were promised 
when making their decisions. 

Professional Accountability 

California’s new accountability sys- 
tem relies on educators to assume 
professional responsibility for pro- 
viding the students under their care 
with the best possible educational 
experience. This includes a commit- 


ment to master a more robust com- 
mon knowledge base upon entry to 
the profession, to share knowledge 
and skills with others, to learn from 
colleagues, and to continuously as- 
sess what is working and what is not. 
In return, the state is committed to 
providing educators with the tools 
and training they need to continu- 
ously monitor their students’ per- 
formance in order to identify areas 
of accomplishment along with areas 
of weakness. For this kind of pro- 
fessional accountability to operate 
effectively, pre- and in-service train- 
ing programs will have to improve 
(and in the latter case, expand) to 
provide the needed knowledge and 
skills; time must be provided for 
in-school learning, assessment, and 
knowledge sharing; and productive 
teacher evaluation systems that pro- 
vide useful, ongoing feedback must 
be developed where they do not cur- 
rently exist. 

In addition, teachers will need ongo- 
ing data about student performance, 
so that they can continually evalu- 
ate and respond to student learning 
needs, and seek additional expertise 
to respond to these needs as appro- 
priate. This will require more than 
simply scores on SBAC tests: It will 
depend on use of formative assess- 
ments, ongoing examination and 
shared analysis of student work, and 
collaboration in planning to address 
these needs. Some of this addition- 
al information and shared planning 


may be acquired through teachers’ 
participation in the scoring of new 
SBAC performance assessments. 

Performance Accountability 

California’s new accountability sys- 
tem is premised on the recognition 
that testing, by itself, does not equal 
accountability, and that a single 
number cannot provide an adequate 
measure of school quality or school 
performance. Local educators are 
therefore required to set goals and 
measure performance on eight sepa- 
rate state priorities in their LCAPs 
(see Figure 2) . For this kind of per- 
formance accountability to operate 
effectively, it will be important for 
local education agencies (LEAs) and 
charter schools to have the capacity 
to access and use the needed data 
and for COEs to have the capacity 
to evaluate it in a reliable, consistent 
manner. To support these activities, 
and to fulfill its own obligations 
under federal and state law to assist 
struggling California schools and 
districts, the state will need to have 
school performance data that can be 
compared across EEAs and can be 
aggregated on a statewide basis. 

Schools and school districts will 
continue to need support and guid- 
ance from the state to be successful, 
even as they redesign themselves as 
learning organizations. That sup- 
port should enable and encourage 
some innovation and risk-taking. 
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Figure 2; Indicators Required in Local Control and Accountability Plans 


Student Achievement 

• Test score gains 

• English proficiency 

• Evidence of college & career 
readiness 


School Climate 

• Suspensions, expulsions 

• Student & professional 
supports 

(student, teacher, and parent 
surveys) 


Student Engagement 

• Attendance 

• Dropout rates 

• Graduation rates 

• Evidence from student surveys 


Parent Involvement 

• Efforts to seek parental input 

• Evidence of parent 
participation 
(parent surveys) 


Other Outcomes 

• Completion of a college or 
career ready pathway 

• Completion of a workplace 
learning or community service 
experience 


Basic Services 

• Teacher misassignment 

• Access to materials 

• Adequate facilities 


Implementation of 
New Standards 

• Implementation of State Board 
of Education-adopted academic 
content and performance 
standards for all pupils, 
including English learners 


Course Access 

• Access to curriculum in the 
core academic subjects, 
STEM, the arts, and physical 
education (student surveys) 


moving beyond old-style account- 
ability methods that encouraged 
conformity and sometimes punished 
inventive ideas. It will also need to 
he conducted with recognition that 
a continuous feedback loop takes 
time to hear fruit, as knowledge 
about successful practices moves 
from classroom or school-wide in- 
novation to evaluation and dissemi- 
nation to curriculum changes and 


professional development that help 
others learn the successful tech- 
niques and to subsequent improve- 
ments in classroom performance on 
a wider scale. 

The agencies of state government, 
including the CDE and the State 
Board of Education (SBE), must 
recognize that local educators can- 
not build a continuously improv- 
ing education system on their own. 


To get to these successful strategies 
sooner and more purposefully, the 
state can identify areas of learning 
where improvement is needed (e.g., 
teaching fractions or persuasive 
writing) and encourage innovation 
and evaluation of strategies to seed 
improvement. To support this kind 
of learning-focused initiative, the 
state’s new accountability system 
must therefore be designed to build 
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school capacity and drive continu- 
ous improvement, as discussed be- 
low. 

Continuous Improvement in 
the Education System 

With the simultaneous implemen- 
tation of LCFF and LCAP, along 
with new standards, curriculum, 
and assessments, schools and school 
districts throughout California have 
a unique opportunity to reconhg- 
ure themselves as learning organiza- 
rions, committed to conrinuous im- 
provement and explicitly organized 
to support experimentation, evalu- 
arion, and organizarional learning. 
LCFF and LCAP give districts the 
resources and flexibility they need 
to experiment with new policies and 
pracrices, but experimentation by 
itself does not lead to organizational 
learning or systemic improvement. 
Too often, in fact, experimentation 
in schools leads only to random acts 
of innovation, which are dependent 
on the initiative of individual educa- 
tors, limited to a single classroom or 
school, and rarely if ever evaluated, 
disseminated, or replicated. What- 
ever gains (or harms) result affect 
only those students who are directly 
involved, while the larger education 
system proceeds undisturbed. 

By contrast, in a continuously im- 
proving system, innovation is a 
thoughtful and deliberate process, 
designed to produce knowledge 


about new programs and practices 

that can be validated and shared. 

Key features of a continuously im- 
proving education system include: 

• Learning supports (materials and 
professional development) for the 
continuous improvement of cur- 
riculum, teaching, assessment, 
and student support strategies; 

• Information systems for keeping 
track of what schools and districts 
are doing and to what effect; 

• Ongoing review of school and 
district efforts and outcomes, 
including self-assessment and re- 
view by experts and peers; 

• Thoughful innovation and 
evaluation, so teachers, schools, 
and school districts experiment 
with promising policies and prac- 
tices in ways that are a) informed 
by existing knowledge about 
those practices, b) designed to 
support serious evaluation of 
their implementation challenges 
and effects, and c) intended to 
support broader adoption of suc- 
cessful approaches and abandon- 
ment of unsuccessful ones; 

• Knowledge dissemination strat- 
egies (through a central reposi- 
tory of research and exemplars, 
convenings, networks, and lever- 
aged supports) so that successful 
practices become widely known 
and supported in their wider 


adoption/ adaptation. 

These features of a continuously im- 
proving system support and interact 
with one another, as suggested in 
Figure 3 below. For example, the 
results of learning supports may be 
partly captured by the information 
system, which can in turn guide the 
ongoing review of strategies and 
outcomes. This review may suggest 
places where interventions are need- 
ed, or where careful experimenta- 
tion with purposeful evaluation can 
make a difference in problem solv- 
ing. The results of useful experi- 
ments and innovations can then be 
shared through a variety of avenues, 
producing a further set of learning 
supports. The cycle of continuous 
improvement proceeds accordingly. 

Learning Supports 

A continuously improving system 
creates an infrastructure for sup- 
porting professional learning and 
building collective capacity across 
classrooms, schools, and districts. 
Coherence is important: Curricu- 
lum, assessment, standards, and 
professional learning opportunities 
should be seamlessly integrated, 
with supports for educator engage- 
ment and learning in each arena. 
In many high-achieving countries, 
for example, the implementation 
of new standards and curriculum 
frameworks often includes time and 
resources for teachers to develop. 
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Figure 3; Elements of a Continuously Improving System 
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test, polish, and share curriculum 
materials and lessons both within 
and across schools. In addition, 
teachers are involved in developing 
and scoring performance assess- 
ments tied to the standards, exam- 
ining and evaluating student work 
so that the standards come alive. 
Professional learning is integral to 
all these activities. 

Effective professional development 
is sustained, ongoing, content- 


focused, and embedded in profes- 
sional learning communities, giving 
teachers the opportunity to work 
over time on problems of practice 
with other teachers in their subject 
area or school.^ Furthermore, it 
focuses on concrete tasks of teach- 
ing, assessment, observation, and 
reflection, looking at how students 
learn specific content in particular 
contexts, rather than emphasizing 
abstract discussions of teaching. 
Equally important, it focuses on 


student learning, including analysis 
of the skills and understandings that 
students are expected to acquire and 
what they are in fact learning, draw- 
ing on collective analysis of student 
work and other relevant data.^ Ex- 
amples of how some countries and 
states have created systemic strate- 
gies for professional learning and 
knowledge dissemination are de- 
scribed in Box 1 below. 
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Box 1 ; Creating Systemic Supports 

for Instructional Improvement 


Some countries have created very 
deliberate approaches to improv- 
ing teaching by systemically dis- 
seminating research and practical 
knowledge about instructional 
strategies. For example, England 
instituted a national training pro- 
gram in “best-practice” teaching 
strategies as part of the National 
Literacy and National Numeracy 
initiative, which provided re- 
sources to support implemen- 
tation of the national curricu- 
lum frameworks. These include 
packets of high-quality teaching 
materials, resource documents, 
and videos depicting successful 
practices. A “cascade” model of 
training — similar to a trainer of 
trainers model — was structured 
around these resources to help 
teachers learn and use productive 
practices. 

The National Literacy and Na- 
tional Numeracy Centers pro- 
vided leadership and training for 
teacher training institutions and 
consultants, who trained school 
heads, lead math teachers, and 
expert literacy teachers who, in 
turn, supported and trained other 
teachers. As more teachers be- 
came familiar with the strategies, 
expertise was increasingly located 
at the local level, with consultants 


and leading mathematics and lit- 
eracy teachers providing support for 
their peers. An additional compo- 
nent of the strategies was designed 
to allow schools and local education 
agencies to learn best practices from 
each other by funding and support- 
ing 1,500 groups of six schools each 
to engage in collaborative inquiry 
and knowledge-sharing.'" 

Similarly, the Australian government 
sponsored a three-pronged Quality 
Teacher Program that has provided 
funding to update and improve 
teachers’ skills and understandings 
in priority areas and enhance the 
status of teaching. Teaching Aus- 
tralia facilitates the implementation 
of nationally agreed-upon teach- 
ing standards, conducts research 
and communicates research find- 
ings, and facilitates and coordinates 
professional development courses. 
The National Projects are designed 
to identify and promote best prac- 
tice, support the development and 
dissemination of professional learn- 
ing resources in priority areas, and 
develop professional networks for 
teachers and school leaders. The 
State and Territory Projects fund a 
wide variety of professional learn- 
ing activities for teachers and school 
leaders, allowing professional de- 
velopment activities to be tailored 


to local needs. These projects in- 
clude school-based action research 
and learning; conferences; work- 
shops; on-line and digital media; 
and training of trainers, school 
project, and team leaders.^ 

Western Australia’s highly success- 
ful Getting it Right (GiR) Strat- 
egy provides specialist teaching 
personnel, professional develop- 
ment, and support to select pri- 
mary schools to improve literacy 
and numeracy outcomes of high- 
needs students, with a focus on 
Aboriginal and other at-risk stu- 
dents. Each school selects a high- 
ly regarded teacher with inter- 
est and expertise in numeracy or 
literacy to be a Specialist Teacher 
(ST), who is then trained through 
a series of seven three-day inten- 
sive workshops over the course 
of their initial two-year appoint- 
ment. The STs work “shoulder to 
shoulder” with teachers in their 
schools for about half a day each 
week with each teacher. The STs 
also monitor and record student 
learning, help teachers analyze 
student learning, model teaching 
strategies, plan learning activi- 
ties to meet the identified needs 
of students, assist with the im- 
plementation of these activities, 
and provide access to a range of 
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resources, sharing expertise and 
encouraging teachers to he reflec- 
tive about their practice.^ Teachers 
show greatly enhanced knowledge 
about how students learn read- 
ing, writing, and mathematics 
and much stronger teaching and 
assessment skills, including their 
ability to use data to identify and 
diagnose students’ learning needs 
and to plan explicit teaching ap- 
proaches to address these needs. ^ 

In Ontario, Canada, supports 
are available for teachers to en- 
gage in school-focused research 
and development. Teachers are 
provided time and support for 
studying and evaluating their own 
teaching strategies and school 


programs, sharing their findings 
with their colleagues, participat- 
ing in conferences, and preparing 
publications.'® School-to-school 
networking strategies are also used 
to share practices; the province has 
leveraged them further by identify- 
ing positive exemplars that schools 
can visit to see what successful re- 
forms look like in action. As Andy 
Hargreaves notes, “Lateral support 
across schools is wedded to positive 
peer pressure as schools push each 
other to higher and higher standards 
of performance.”" 

Similarly, an initiative in England 
to improve 300 struggling schools 
networked them with one another 
in small groups, provided them 


with technical assistance and sup- 
port from mentor schools, and 
gave them a small discretionary 
budget they could spend to sup- 
port their efforts. Schools were 
also given a practitioner-generat- 
ed list of strategies that had pro- 
duced short, medium, and long- 
range improvements. More than 
two-thirds of these “exceptionally 
energized” schools experienced 
gains over the next two years at 
rates double the national average, 
“without the characteristic man- 
dates and prescriptions that had 
characterized English reforms be- 
fore this point.”'^ 


Information Systems 

To support continuous improve- 
ment, information systems must of- 
fer transparent, accessible data that 
are organized and reported in ways 
that enable educators, agencies of 
government, and other stakehold- 
ers to examine what is happening 
over time at various levels of detail 
(for example, with disaggregation 
by student groups, by schools, etc.). 
These systems should support the 
planning and evaluation that guide 
continuous improvement. Informa- 
tion systems ideally provide read- 
ily available data about a range of 
student outcomes, including atten- 


dance, progression through school, 
graduation, and measures of learn- 
ing, along with data about students’ 
experiences that may be associated 
with these outcomes (e.g., measures 
of access to qualified teachers, high- 
quality coursework and program 
opportunities, and positive school 
climate, among others). 

Ongoing Review 

Regular review strategies are key to 
continuously improving systems. 
Reviews — whether of individual 
educators, schools, or districts — 
should incorporate a range of data 
about what is happening and to 


what effect into more comprehen- 
sive, qualitative examinations that 
can drive planning for improve- 
ment. Review strategies should 
include regular self-assessment and 
reflection using rubrics or other 
tools that clarify important areas of 
work and expectations about perfor- 
mance, as well as review by experts 
and peers. 

If continuous improvement is to 
take root in California’s education 
system, ongoing review should be- 
come a predictable and welcome 
activity both within and across 
schools. If schools are to become 
more responsible and responsive. 
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they must, like other professional 
organizations, make evaluation and 
assessment part of their everyday 
lives. Just as hospitals have stand- 
ing committees of staff which meet 
regularly to look at assessment data 
and discuss the effectiveness of each 
aspect of their work (a practice re- 
inforced hy their accreditation re- 
quirements) so schools must have 
regular occasions to examine their 
practices and effectiveness. 

School-level accountahility can he 
supported hy external reviews cou- 
pled with self-assessment. In many 
other countries, school inspections 
are conducted hy trained experts, 
usually highly respected former 
practitioners, who evaluate schools 
hy spending several days visiting 
classrooms, examining random 
samples of student work, and in- 
terviewing students about their un- 
derstanding and their experiences. 
They also look at objective data such 
as test scores, graduation rates, and 
the like.'^ An Americanized version 
of the inspectorate system has been 
designed by former members of the 
British inspectorate in consultation 
with U.S. educators. This hybrid 
system combines peer reviews con- 
ducted by teams of outside educa- 
tors with expert reviews in a process 
that includes school self-assessment. 
It has been used in several states and 
cities, including New York, Rhode 
Island, and Chicago.'^ 


Thoughtful Innovation and 
Evaluation 

Continuous improvement relies in 
part on systematic experimentation 
with new policies and practices, to 
identify those that show promise. 
California’s new accountability sys- 
tem gives educators the freedom 
and flexibility to experiment, but 
freedom and flexibility alone are 
not enough to support innovation. 
The state’s previous accountability 
system was strongly guided by state 
and federal regulations, which pro- 
duced strong incentives for local 
educators to focus on compliance 
and minimize risk. Changing this 
orientation will require positive ac- 
tion to shift the incentives in the 
accountability system to encourage 
rather than discourage experimenta- 
tion, in the full knowledge that not 
all innovations are successful. 

Continuous improvement re- 
quires that new policies and prac- 
tices be introduced deliberately 
and thoughtfully, in ways that can 
support rigorous evaluation and 
policy learning. This is first of all 
a question of policy design. Rath- 
er than introducing new practices 
and policies simultaneously for all 
students, as is typically done now, 
schools and school districts should, 
when possible, plan for the staged 
implementation of pilot projects, 
policy experiments, or even ran- 
domized control trials, which can 


inform efforts to learn which in- 
novations show promise and which 
should be abandoned. Identifying 
which innovations promise benefits 
for students requires knowledge of 
existing research and professional 
practice, as well as nuanced evalu- 
ation to learn which ones work, for 
which students, and under which 
sets of circumstances. Developing a 
repository of evidence about effec- 
tive policies and practices is a key 
role for the state in California’s new 
accountability system. 

In addition to the information that 
can be provided by the state’s data 
system, continuous improvement 
will require that schools and school 
districts collect and analyze data 
needed to monitor and evaluate lo- 
cal innovations. This may require 
school districts and CSAs to invest 
in local data systems that can track 
students’ participation in specific 
programs and activities over time in 
order to distinguish those that are 
moving students toward college and 
career readiness from those that are 
not. Careful evaluation of alterna- 
tive policies and practices is essen- 
tial if California is to move beyond 
“random acts of innovation” toward 
a system that is capable of policy 
learning and continuous improve- 
ment. 
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Knowledge Dissemination 
Strategies 

A continuously improving system 
needs well developed strategies for 
sharing knowledge about both in- 
structional practice and education- 
al policies and programs. At the 
level of day-to-day practice, qual- 
ity teaching depends not just on 
teachers’ knowledge and skill but 
on the environments in which they 
work. Schools should be designed 
for knowledge sharing that enables 
them to offer a coherent curriculum 
focused on higher order thinking 
and performance across subject ar- 
eas and grades, with high-quality in- 
structional norms and practices that 
are also shared. This requires time 
for teachers to work intensively with 
students to accomplish challenging 
goals; opportunities for teachers to 
plan with and learn from one anoth- 
er; and regular occasions to evaluate 
the outcomes of their practices. 

To sustain continuous improvement 
in California’s education system, in- 
centives and opportunities must be 
structured in ways that promote 
collaboration and knowledge-shar- 
ing across schools, school districts, 
COEs, and CSAs. This approach 
has been the primary strategy for 
improvement in Ontario, Canada, 
where ongoing evaluation and in- 
quiry into practice are encouraged 
and supported within and across 
classrooms, across schools partnered 


within regions, and within the sys- 
tem as a whole'^ (see Box 1 above). 

If California aims to build a con- 
tinuously improving education sys- 
tem, the state will have to develop 
and support platforms and channels 
that make it easy for local educators 
to share information about promis- 
ing practices and to learn from one 
another. This can happen through 
dissemination of research and best 
practice documentation, network- 
ing of schools and districts, and lev- 
eraged supports for adoption and 
adaptation of successful practices 
among educators, schools, and dis- 
tricts. This kind of policy learning 
is a necessary condition if local in- 
novations are to spread beyond the 
schools and classrooms where they 
originate and go to scale. State 
agencies including the CDE and the 
CCEE have key roles to play in sup- 
porting the compilation, curation, 
and dissemination of useful and ac- 
tionable information to educators 
throughout California. More in- 
tensive knowledge supports may be 
needed for schools and districts that 
are struggling to provide a mini- 
mally adequate education, and these 
will need to be part of the plan. 

How California Can Create 
a System for Continuous 
Improvement 

California’s new accountability sys- 
tem seeks to ensure that schools have 


the appropriate resources, profes- 
sional competence, and incentives 
to offer every child a high-quality 
education. The new system focuses 
on a broader set of outcomes than 
in the past, in order to reflect more 
clearly what students need in order 
to be prepared for college, career, 
and citizenship. It is grounded in 
the following principles: 

• Meaningful learning should sup- 
port acquisition of the knowl- 
edge and skills that students need 
to succeed in today’s world: the 
ability to apply complex knowl- 
edge to solve problems, collabo- 
rate, communicate, inquire and 
learn independently, build rela- 
tionships, and be resilient and 
resourceful. 

• Trust and responsibility should be 
achieved by strengthening profes- 
sional accountability along with 
parental involvement and acces- 
sible data, so that educators and 
parents have the best available 
knowledge to make decisions that 
serve each child well. 

• Transparency should ensure that 
information about school deci- 
sions, inputs, and outcomes is 
readily available to support con- 
tinuous improvement and ac- 
countability. 

• Multiple measures should inform 
all decisions about students, 
teachers, and schools. These 
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should capture the many aspects 
of education valued hy parents, 
educators, and community and 
should he evaluated through sys- 
tems of review, judgment, and in- 
tervention that support continuous 
improvement. 

• Reciprocity and subsidiarity should 
guide state-local relationships. 
Each level of the system should 
he held responsible for the con- 
trihutions it must make to sup- 
port learning for every child. The 
state is responsible for providing 
adequate and equitable resources, 
while local districts must allocate 
resources intelligently to meet stu- 
dents’ needs. 

• Community engagement should 
be the result of local agency that 
allows schools and districts, with 
community input, to make ap- 
propriate decisions on behalf of 
their students. 

• Collaboration and coherence with- 
in schools, across schools within a 
more rationally designed system, 
and between P12 and higher ed- 
ucation, should enable the edu- 
cational system to operate more 
effectively in meeting the state’s 
educational needs. 

• Creativity and flexibility should be 
encouraged in order to meet the 
demands of the future and the 
full range of student needs. Mul- 
tiple pathways to success, featur- 


ing relevant, engaging learning, 
should be available to students to 
enable them to access a produc- 
tive future. 

For these goals to be accomplished, 
California will need to create a sys- 
tem that operates differently from 
the system we have today in many 
significant respects. California’s new 
accountability system is designed to 
guide the state’s schools and school 
districts onto a path of continuous 
improvement. To do this, it will 
need: 

• a more highly developed system 
of learning supports; 

• an integrated and usable infor- 
mation system to guide planning 
and evaluation; 

• processes that employ these data 
in ongoing reviews that incorpo- 
rate qualitative information, self- 
and peer-assessment, and expert 
judgment to support continu- 
ous improvement for educators, 
schools, and districts; 

• thoughtful innovation and evalu- 
ation processes that can produce 
knowledge about what works and 
under what circumstances; and 

• knowledge-sharing vehicles with- 
in and across schools. 

In what follows, we focus on two key 
pillars that can support these ele- 
ments of a continuous improvement 


system. First, we describe what we 
believe will be needed to produce a 
usable information and reporting 
system that can enable educators 
and state agencies to better organize 
learning supports, ongoing review, 
and useful evaluation. Second, we 
describe the role the CCEE could 
play in creating general supports for 
learning, knowledge sharing, and 
evaluation, as well as for direct in- 
tervention and improvement in the 
work of those schools and districts 
that are struggling to provide an ad- 
equate education. These two pillars 
are linked, as a productive role for 
CCEE will depend in substantial 
measure on a high-quality informa- 
tion system. 

An Information and 
Reporting System to 
Support Continuous 
Improvement 

An important key to a sustained 
process of continuous improvement 
is the regular review of data to guide 
diagnosis of local strengths and 
weaknesses and to identify strategies 
and practices to support improved 
performance. Educators are expect- 
ed to make use of this information 
to identify areas where their students 
are not meeting expectations and to 
adapt their professional practices 
to address local weaknesses. The 
broader school community is simi- 
larly expected to hold educators ac- 
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countable for the wise and effective 
use of resources and for targeting re- 
sources to support the achievement 
of local goals. Over time, these 
mutually reinforcing accountability 
mechanisms are intended to pro- 
duce a virtuous cycle of continuous 
improvement in the performance of 
schools and students. 

Until recently, the primary measure- 
ment tool used to rank schools and 
identify them for intervention has 
been the Academic Performance In- 
dex (API). Although it was origi- 
nally intended to incorporate a wide 
range of measures, the technical dif- 
ficulties of mixing measures with 
different scales and meanings have 
precluded the use of much more 
than test scores in the API. Even if 
the effort to incorporate additional 
measures were to prove technically 
feasible, however, the results would 
not provide useful information for 
a continuous improvement system: 
Tossing a mix of apples, oranges, 
spinach, and chocolate chips into a 
blender may produce an edible mix- 
ture, but once the ingredients are 
blended it is virtually impossible to 
distinguish them from one another 
or to know how each one influenced 
the final result. A single index that 
blends multiple measures of school 
performance is a poor tool for guid- 
ing planning or improvement. 

In addition, as businesses and many 
other organizations are well aware. 


boosting scores on a single measure 
can sometimes have a negative im- 
pact on other important outcomes. 
For example, businesses recognize 
that an exclusive focus on short-run 
profits can undermine investments 
in innovation or in building up a 
loyal customer base that is necessary 
for longer term success. Similarly, 
a single-minded focus on test scores 
can create incentives for educators 
to counsel struggling students out 
of high schools and into continua- 
tion schools or GED programs or 
to leave school entirely. This may 
boost test scores in some schools, 
but at the cost of lowering gradua- 
tion rates in the system as a whole 
and failing a group of students. 

Finally, the use of norm-referenced 
school rankings, including perfor- 
mance deciles, to guide policy has 
proven increasingly problematic, 
because it does not provide infor- 
mation about or credit schools with 
their success in reaching a criterion 
or benchmark. Reliance on norm- 
referenced rankings merely evaluates 
schools in relation to one another 
on a single indicator. In the case of 
California’s API, that indicator was 
more highly correlated with student 
socioeconomic status than with any 
variable or factor that districts or 
schools could affect. 

In California’s new accountability 
system, the state’s efforts to create 
a productive, aligned system for 


improvement should aim for a uni- 
fied approach focused on the eight 
priorities that frame the ECAP (see 
Figure 2). Rather than maintain 
competing systems and measures, 
the API and the current School Ac- 
countability Report Card (SARC) 
should be replaced with a dash- 
board of measures that is designed 
to help organize data for planning, 
public information, and improve- 
ment by reporting progress on the 
state’s eight priorities. These mea- 
sures should reveal how schools and 
districts are improving and how 
they are doing in relation to criteria 
for performance in different areas, 
rather than seeking to rank them on 
a single measure. 

As we discuss below, reporting on 
multiple measures of school perfor- 
mance can help to identify schools 
that need assistance from COEs and 
the CCEE as well as signal the ar- 
eas where assistance is likely to have 
the greatest impact. It can also help 
identify schools that are succeeding 
and provide opportunities for oth- 
ers to learn from their success. This 
will help schools focus their efforts 
on shared goals without subjecting 
them to conflicting expectations or 
requirements. 

Unless a single, coherent system of 
accountability measures is adopted, 
California runs the risk of perpetu- 
ating a set of incompatible practices 
that confuse the field; waste educa- 
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tors’ energy as they respond to mul- 
tiple reporting requirements; and 
undermine clarity, trust, and a local 
capacity to mobilize scarce resources 
for positive change. Schools and 
the state as a whole would suffer if 
the CDE were to continue using a 
different measure to idenrify schools 
for intervention under its federal 
and existing state obligations from 
that used by COEs, CSAs and the 
CCEE to examine and support con- 
tinuous improvement in schools and 
school districts. Instead the state 
should develop a coherent approach 
that places the state’s eight priorities 
at the heart of a unified accountabil- 
ity system that is augmented with 
local measures reflecting additional 
community goals and priorities. 

Moving to a System of Multiple 
Measures. The ECAP process is 
based on a dashboard of indica- 
tors that are intended to measure 
status and growth on state priori- 
ties. The use of multiple measures 
is much more informative than us- 
ing a synthetic index for planning 
and improvement efforts. Eike the 
dashboard on a car — which pro- 
vides indicators of speed, distance 
traveled, fuel, fluids, tire pressure, 
and more — the combination of 
measures provides information 
about where to look further in order 
to figure out how things are work- 
ing and what may need attention. 
The system is premised on multiple 
measures, which may ultimately 


include periodic state standardized 
measures and assessments of student 
performance to augment and verify 
the results of more robust measures 
and assessments that are used at the 
local level. 

Properly conceptualized, the dash- 
board of indicators produced under 
the ECAP could give educators and 
their communities much of the in- 
formation that they need to identify 
areas of strength and areas of con- 
cern and to then target resources 
toward those areas where improve- 
ment is needed. In order to perform 
this function, and to support the 
work of the COEs and CSAs that 
are to review the work of local com- 
munities, as well as the work of the 
CCEE, some common approaches 
to reporting and recording infor- 
mation will be vitally important. 
COEs, CSAs, and the CCEE will 
need a way to examine individual 
school and district data that allows 
multiple comparisons (e.g., to the 
same school over time, to schools 
and districts in the same county, to 
schools and districts serving similar 
students, and so on). Support for 
these kinds of comparisons is not 
currently envisioned in the ECAP 
reporting system, and will need to 
be developed if the proposed goals 
are to be reached. 

A statewide tool for helping to col- 
lect and report data will be needed 
for a number of reasons: 


• To assist local schools and dis- 
tricts, many of which are cur- 
rently having difficulty defining 
and producing indicators for a 
number of the ECAP categories; 

• To produce evidence about suc- 
cesses that others across the sys- 
tem can learn from and replicate; 

• To support evaluation of the out- 
comes of changes in practice or 
policy; 

• To enable parents and the public 
to assess the performance of their 
schools and districts in relation to 
others; 

• To enable COEs and CSAs, as 
well as the CCEE when it is ac- 
tivated, to fairly assess the per- 
formance of schools and districts 
relative to their past performance 
and to their peers in order to bet- 
ter assist them in improving; 

• To allow aggregation of the data 
for state policymakers who will 
want to know about successes 
and challenges for the system as 
a whole, or for certain age ranges 
or student groups, in order to in- 
form state policy development. 

Currently, the burden of producing 
and reporting data for the ECAP 
rests solely with local districts and 
charter schools, and ECAP require- 
ments have not been integrated with 
other state data systems, including 
CDE’s on-line accountability re- 
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porting tool, or SARC. In the past 
year, the state took an important 
step toward a more coherent sys- 
tem of accountahility reporting hy 
rearranging the SARC template to 
display data in ways that are better 
aligned with the LCAP state priori- 
ties, hut there are many LCAP pri- 
orities for which data are still not 
provided. Other data systems at the 
CDE are also poorly aligned with 
the LCAP. The CDE does not cur- 
rently have a process to send data to 
local districts in a form that they can 
directly use for ECAP planning and 
reporting, nor does it have a process 
to organize locally reported data in 
ways that allow the data to he ag- 
gregated and reviewed at the county 
or state levels. 

As the state moves toward a more 
aligned and better integrated ac- 
countability system, an additional 
policy lever is the evaluation rubric 
that will be used in examining and 
assessing the ECAPs. The rubric is 
to include standards for school dis- 
trict and individual school perfor- 
mance as well as expectations for 
improvement for each of the eight 
state priorities. The purposes of 
this rubric are to: (1) assist school 
districts, COEs, and charter schools 
in assessing their own performance, 
including their strengths, weakness- 
es, and areas for improvement; (2) 
assist COEs in determining when 
school districts and charter schools 
need assistance; and (3) assist the 


SPI, SEE, and CSAs in determining 
when they should provide a school 
district or charter school with assis- 
tance or take other appropriate ac- 
tion authorized by the law, such as 
intervention or charter revocation. 

This rubric could organize the re- 
porting of and standards for state 
measures that could be part of the 
ECAPs as well for local measures. 
By integrating the format of the ru- 
bric with the SARC and the on-line 
reporting system that currently ex- 
ists for each school and district, the 
state could provide the public with 
transparent, accessible data, while 
saving EEAs time and resources they 
would otherwise need to spend to 
develop all their own measures. 

A Statewide Reporting Took An 

on-line statewide reporting tool 
to support ECAPs could incorpo- 
rate the requirements of the rubric 
to set standards or benchmarks for 
each of the indicators and could 
provide available state data in many 
categories. Then local schools and 
districts could draw from this tool 
and add their own indicators and 
data for the remaining areas that are 
not amenable to comparable data or 
where local goals have been includ- 
ed in the ECAP. 

An example of what such a tool could 
look like is included in Appendix B. 
It is modeled on the Alberta (Cana- 
da) Results Report, which is an on- 


line tool that contains data for that 
province’s seven sets of indicators in 
its multiple measures accountability 
system. This tool is publicly avail- 
able, and those who are interested 
can click on any of the indicators to 
see further detail on the underlying 
data that produced the measure.'*’ 
Eocal districts can use these data, 
along with their own locally devel- 
oped data, when they prepare their 
Alberta Education Results Reports, 
which, like the ECAPs, allow them 
to summarize their accomplish- 
ments and set goals as well as report 
results.'^ 

There are many ECAP areas for 
which statewide data are already 
available (e.g., state test scores in 
English language arts (EEA), math, 
science, and English language pro- 
ficiency; attendance; persistence 
rates; graduation rates; a-g comple- 
tion rates; and assignment of cre- 
dentialed teachers). In some cases, 
these would have to be assembled 
from various departments or agen- 
cies of state government. There are 
a number of other areas where the 
state could readily develop or col- 
lect data from local districts in com- 
mon ways (e.g., completion of an 
approved career technical education 
(CTE) course sequence; student 
completion of college-ready stan- 
dards on Advanced Placement or 
International Baccalaureate tests, or 
through dual credit courses; school 
climate and opportunity to learn in- 
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dicators from surveys, as discussed 
below). Where local definitions and 
data are needed, they could be add- 
ed to this foundation. 

Surveys. The dashboard could be 
informed by a set of student, teach- 
er, and parent surveys that include 
some common questions across the 
state in addition to locally identified 
questions. The surveys would pro- 
vide an efficient way to collect evi- 
dence about school climate, student 
engagement, parent involvement, 
access to basic services, implemen- 
tation of the CCSS, and course ac- 
cess, without creating burdensome 
reporting requirements for schools 
and districts. 

Currently, the California School 
Climate, Health, and Learning Sur- 
vey System, operated by the CDE, 
provides surveys of pupils, parents, 
and school staff regarding school cli- 
mate and other issues that districts 
can access for a fee. The CORE 
districts also have created a teacher, 
parent, and student survey to pro- 
vide information for their multiple 
measures accountability system. If 
a small number of items were iden- 
tified for a statewide on-line survey 
that districts could freely access and 
to which they could add their own 
items, this would provide a service 
to districts that are unable to mount 
their own effort and would allow for 
more common measures in some 
domains. In addition to indicators 


of school climate (including safety 
and connectedness), a set of surveys 
could contribute data about par- 
ent involvement, the availability of 
a rich curriculum, and even CCSS 
implementation, as illustrated in 
Figure 2 above. 

Use of Data. Organized in this 
way, the ECAP dashboard of indi- 
cators could provide annual data 
that indicate, for each measure, how 
the school or district is performing 
overall and in relation to prior years, 
disaggregated for groups of students 
(by race/ethnicity, language back- 
ground, special education status, 
and poverty). These state and local 
data can provide the grist for con- 
tinuous improvement processes at 
the local school and district levels, as 
well as for the provision of analysis 
and support by COEs, CSAs, and as 
warranted, by the CCEE. 

Evaluation across multiple mea- 
sures can suggest how schools and 
districts are doing and whether they 
are improving. If a number of con- 
cerns are apparent, the school or 
district could be identified for fur- 
ther diagnostic review, evaluation, 
and an improvement process. The 
CDE should align any process it is 
required to use for federally required 
identification of schools for inter- 
vention with this process, so that 
the system is coherent. As described 
in Appendix A, the CORE districts, 
under their federal Elementary and 


Secondary Education Act (ESEA) 
waiver, have designed a multiple 
measures system compatible with 
the ECAP, which adds additional 
local measures and can be used for 
federally required identification and 
support for struggling schools. 

A state data tool that supports lo- 
cal districts should enable and sup- 
port the ECAP process, but it should 
not become the process, nor should 
it drive budgeting and goal-setting. 
The local strategic planning and 
budgeting process that the ECAP 
is meant to stimulate and inform 
should be grounded in local con- 
texts and priorities, not determined 
by a data tool. The ECAP is intend- 
ed to guide how funds are allocated 
to achieve results. It is critically im- 
portant that the ECAP not become 
a checklist that ticks off whether 
districts put money in every cat- 
egory defined by the state priorities. 
That is not the hope or expectation 
of the ECAP process. Eocal districts 
can be accountable for results only 
if they can use their best judgment 
about how to achieve them. 

Technical Assistance 

The primary use of the new ECAP 
evaluation rubrics will be to provide 
guidelines against which schools 
and districts can assess their own 
progress and performance to guide 
ongoing improvements. In some 
cases, however, local dynamics 
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will not be sufficiently powerful to 
launch this cycle, and schools and 
school districts will need additional 
help from outside to be successful. 
Some schools and school districts 
will make uninformed decisions, 
and others will lack the capacity to 
transform struggling schools. Local 
educators will need external support 
to help them build internal capac- 
ity and foster a local culture that 
can support continuous improve- 
ment in the performance of schools 
and students. Under the LCFF, the 
primary responsibility for provid- 
ing technical assistance and support 
to local schools and districts resides 
with COEs and CSAs, while the ul- 
timate responsibility rests with the 
CCEE. 

Intervention by a COE or CSA is 
not intended to be punitive; rather, 
the goal is to help diagnose local 
obstacles to success and to provide 
technical assistance aimed at build- 
ing local capacity to overcome those 
obstacles. Schools and school dis- 
tricts that recognize their own lack 
of progress may decide to request 
technical assistance or support from 
the COE or CSA. In cases where 
the school or school district is mak- 
ing less than satisfactory progress to- 
ward accomplishing its stated goals 
over an extended period of time, 
however, the COE or CSA will be 
required to intervene. 

Many of California’s COEs have 


experience in providing this kind of 
technical assistance to local schools 
and school districts, and many dis- 
tricts look to the COE as their pri- 
mary source of help when they run 
into difficulties.*® Many COEs now 
provide professional development 
for teachers and other educators on 
a fee-for-service basis, and COEs 
have played a lead role in the con- 
stitution and deployment of tech- 
nical assistance teams, as in School 
Assistance and Intervention Teams 
(SAIT) and District Assistance and 
Intervention Teams (DAITs) . 

However, not all COEs or CSAs 
now have the expertise or capacity to 
provide the support that the schools 
and school districts under their ju- 
risdiction may need. As the ECAP 
process matures, COEs will need to 
build their capacity to assist local ed- 
ucators, which may require substan- 
tial organizational redesign in some 
counties. One strategy for enhanc- 
ing local capacity would be to work 
on a regional basis to share resources 
and responsibilities. There may be 
opportunities for COEs to special- 
ize in particular areas (e.g., English 
language development, technology), 
or to coordinate services in a region- 
al consortium. Regional consortia 
can significantly expand the variety 
and quality of services that COEs 
can provide to local educators, but 
fostering the level of cooperation 
and coordination across schools, 
districts, and counties necessary will 


pose a significant challenge to local 
leaders. Developing and orchestrat- 
ing productive technical assistance 
models across the state could be one 
of the roles of the new CCEE. 

The California Collaborative 
for Educational Excellence 

The cornerstone of California’s new 
accountability system is the Califor- 
nia Collaborative for Educational 
Excellence, not because it can or 
will do most of the work required to 
support continuous improvement 
in the system, but because it is the 
agency that is ultimately responsible 
for making the system work. Un- 
der the ECEE statute, the CCEE’s 
main responsibility is to help devel- 
op and implement strategies to im- 
prove local performance in schools 
and school districts where local or 
regional action by the COE or CSA 
has failed to launch a cycle of con- 
tinuous improvement. This entails 
two main tasks, as outlined below. 

Eirst, the CCEE must provide di- 
rect assistance to schools, school 
districts, COEs, and CSAs that are 
falling short of their goals and ob- 
ligations, including both those that 
are identified by the Superintendent 
of Public Instruction (SPI) and those 
that request assistance on their own. 
To fulfill this responsibility, the 
CCEE will have to conduct skillful, 
diagnostic investigations into the 
reasons why schools are not meeting 
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performance expectations. These 
investigations should include ex- 
tended visits to schools and consul- 
tation with local stakeholders, and 
not just a review of test scores. Site 
visit teams will produce diagnos- 
tic reports that identify weaknesses 
and problems and suggest context- 
sensitive strategies for remediation, 
with the goal of strengthening local 
capacity for organizational learning 
and continuous improvement. 

Second, to do its work effectively in 
schools that operate across the full 
spectrum of California’s very diverse 
educational system, the CCEE must 
huild its own capacity to compile 
and evaluate information about 
practices, tools, and resources that 
show promise, and to share this 
information with COEs, CSAs, 
school districts, and schools. Hiring 
expert practitioners to staff site visit 
and school intervention teams is es- 
sential, but those experts and the lo- 
cal teachers and administrators they 
work with must have access to a re- 
liably curated and easily accessible 
source of information about prom- 
ising innovations in order to keep 
their knowledge up to date in what 
is a diverse and constantly chang- 
ing educational landscape. Elseful 
information may come from other 
practitioners (e.g., via the CDE’s 
“Brokers of Expertise”'^ and/or 
similar on-line repositories) or from 
more formal evaluation research. 


The closest analog to the work of the 
CCEE is the Fiscal Crisis and Man- 
agement Assistant Team (FCMAT), 
which is administered by the Kern 
County Office of Education (see 
Box 2 below) . The responsibilities of 
the CCEE are similar to those faced 
by FCMAT in some respects, but 
the challenges involved in diagnos- 
ing school and district performance 
on multiple indicators and provid- 
ing useful and actionable advice on 
strategies for improvement are vastly 
more complex (though perhaps no 
easier) than the challenge of balanc- 
ing district budgets. 


BOX 2; Fiscal Crisis and 
Management Assistance 
Team 

In 1991, the California legis- 
lature enacted Assembly Bill 
1200, which provided COEs 
with increased oversight respon- 
sibilities for the fiscal and man- 
agement health of their local 
school districts and authorized 
the creation of the Fiscal Cri- 
sis and Management Assistance 
Team (FCMAT). The mission 
of FCMAT is to provide proac- 
tive and preventative fiscal and 
management assistance to EEAs, 
COEs, charter schools, and 
community colleges. Over the 
years, FCMAT’s responsibilities 
have grown to include preparing 
comprehensive assessments and 
recovery plans, providing fiscal 
training to school business offi- 
cials, and developing and main- 
taining two statewide informa- 
tion systems. It is estimated 
that in 2012-2013, 88% of FC- 
MAT’s work entailed providing 
management assistance and only 
12% of its studies involved fiscal 
crisis intervention work.^' FC- 
MAT may be asked to provide 
fiscal crisis or management assis- 
tance by an TEA, COE, charter 
school, community college, or 
assigned by the SPI or the Eeg- 
islature. 
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FCMAT is authorized by the 
Legislature to charge education 
agencies a daily rate for its on-site 
assistance activities when the as- 
sistance is requested by the educa- 
tion agency. FCMAT’s governing 
board sets the billable rate, which 
was $500 per consultant per day 
plus expenses in 2014.^^ Other ac- 
tivities not conducted at an edu- 
cation site, such as report writing, 
are paid for from FCMAT’s an- 
nual appropriation for fiscal and 
management assistance services. 
When FCMAT services are need- 
ed due to a fiscal emergency or the 
education agency receiving a state 
emergency loan, the state rather 
than the education agency pays 
the cost of services. The think- 
ing is that if education agencies 
are not overburdened by the cost 
of receiving services to improve 
practices and system functioning, 
they are more likely to take pro- 
active steps toward implementing 
difficult changes. 

One of the strengths of the FC- 
MAT service delivery model stems 
from the fact that it is an inde- 
pendent and external entity, so it 
can provide an unbiased and ob- 
jective assessment of professional 
and legal performance standards 
with a high level of credibility. 


This is often useful in getting nec- 
essary changes made by local school 
boards.^^ FCMAT’s collaborative 
stance in working with education- 
al institutions is another strength. 
When responding to a request for 
assistance or an assigned review, FC- 
MAT works closely with the educa- 
tion agency to define the scope of 
work, conduct on-site fieldwork, 
and develop a written report with 
findings and recommendations to 
resolve the threats to the agency’s 
fiscal and management health.^^ 

While some think FCMAT could 
benefit from greater authority, either 
to mandate its recommendations or 
to report on agencies’ implementa- 
tion of the recommendations, others 
note that FCMAT’s narrow scope 
of authority may bolster the educa- 
tion community’s perception of it 
as a helping entity rather a compli- 
ance monitor. Therefore, educa- 
tion agencies may be more willing 
to seek assistance from FCMAT 
prior to finding themselves in fis- 
cal and management crises, and the 
education staff may be more willing 
to share sensitive information and 
practices that help FCMAT inves- 
tigate the issues and ultimately de- 
velop more effective and applicable 
recommendations.^^ 


A governing board and the Kern 
County superintendent’s office 
manage the oversight of FCMAT. 
FCMAT’s governing board of di- 
rectors provides policy direction 
and operational guidance to the 
FCMAT and also monitors its 
progress. The governing board 
consists of one county superin- 
tendent and one district superin- 
tendent from each of the state’s 1 1 
service regions, the Chancellor of 
the Community College system 
(or his designee), a representative 
from the community college dis- 
trict governing board appointed 
by the chancellor, and a represen- 
tative from the CDE. A signifi- 
cant oversight responsibility per- 
formed by the governing board 
is developing criteria and priori- 
tizing requests for assistance that 
get submitted to FCMAT. This 
ensures that “FCMAT is kept free 
from real and perceived interfer- 
ence or conflicts of interest that 
might impair its independence 
and objectivity.”^'^ Similar mea- 
sures may help to ensure the in- 
dependence and objectivity of the 
CCEE. 
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Organizing the state’s education 
system to support continuous im- 
provement will ultimately require 
California to build (or rebuild) its 
capacity to generate, validate, and 
share information about promising 
policies and practices. Specifically, 
the state will need to make a com- 
mitment to sponsoring evaluation 
research and generating new knowl- 
edge about what works, and under 
what circumstances, in order to 
improve performance in California 
schools and school districts. In ad- 
dition, in a state that has eliminated 
most of the infrastructure for pro- 
fessional learning, the state will have 
to figure out what kind of learning 
supports it can contribute to the 
range of learning needs schools and 
districts will present. One espe- 
cially urgent task is to provide guid- 
ance to schools and school districts 
about the quality and alignment of 
instructional materials and profes- 
sional development programs that 
claim to advance the goals of CCSS 
implementation.^^ 


Below we discuss how the CCEE, 
in concert with COEs, CSAs, and 
other organizations, might work 
to establish the conditions for con- 
tinuous improvement in California’s 
education system in the following 
areas: 

• Diagnosis of school/ district func- 
tioning through a School Quality 
Review (SQR) process 

• Assistance for improvement 

• Knowledge organization and 
sharing 

• Support for evaluation 

• Support for learning 

Over time, some of these additional 
responsibilities might come to be 
provided under the auspices or with 
the support of the CCEE. Initial- 
ly, it is essential that the CCEE be 
given the time and resources it will 
need to accomplish its core goals of 
providing diagnostic reviews and 
technical assistance for the schools 
and school districts that are most in 


need of help. Other potential roles 
should be considered for phase-in 
over time. 

Diagnostic Review 

In order to support continuous im- 
provement in the performance of 
the state’s schools and students, the 
CCEE will need a process of skillful 
evaluation, diagnosis, and planned 
support for individual schools or 
districts it is working with. In ad- 
dition, the state as a whole could 
benefit from a process that allows 
schools to self-evaluate and receive 
feedback about their practices and 
progress toward their goals. 

These objectives could be accom- 
plished through the establishment 
of a SQR system operated by the 
CCEE, which is used for all schools 
that are the focus of CCEE’s at- 
tention, and which could be made 
available to a wider range of inter- 
ested schools or, as in some states, 
for all schools on a periodic basis 
(see Box 3). 
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BOX 3; The School Quality Review 


School Quality Reviews derive 
from the English school inspec- 
tion system that spread to many 
other Anglophone countries. The 
British inspectorate, founded in 
the 1840s, conducts systematic, 
routine inspections of all English 
schools, which gather observa- 
tional and interview data on 27 
dimensions of school effective- 
ness. The multi-day school in- 
spections are conducted hy paid 
education professionals recruited 
for their expertise and experi- 
ence. Many were former educa- 
tors with a proven track record of 
successful leadership who possess 
content knowledge expertise such 
as specialization in mathematics 
or data analysis. Inspectors go 
through a rigorous hiring process 
that includes reference checks, in- 
terviews, presentations, and per- 
formance tasks. There are 250 of 
Her Majesty’s Inspectors (HMIs), 
who can also access assistance 
from a team of additional inspec- 
tors with similar successful track 
records in education. The HMI 
reports directly to Parliament on 
its evaluations of schools.^® 

The SQR approach was devel- 
oped in the United States in con- 
cert with members of Her Maj- 
esty’s Inspectorate by educators 
who wanted to create strategies 
for helping schools examine their 


practice in order to improve. The 
approach has been used for many 
years for all schools in Rhode Island 
and New York City as well as peri- 
odically in other states or districts — 
often focused primarily on schools 
that are struggling.^^ 

The SQR includes many of the same 
components included in English 
school inspections, such as gather- 
ing first-hand evidence of teaching 
practice through classroom obser- 
vations and conducting interviews 
with students and school staff in or- 
der to make professional judgments 
about the school’s quality and effec- 
tiveness. However, U.S. versions 
of the review have brought together 
several elements that have not been 
joined before in most education 
policy systems: robust data, educa- 
tional expertise, and peer review. 

Eike the Inspectorate model used in 
England and many other countries, 
it is guided by experts (often highly 
accomplished teachers and adminis- 
trators) who are deeply knowledge- 
able about practice and well trained 
in how to conduct a diagnostic in- 
quiry into school practices and their 
relationship to the nature and quali- 
ty of student learning. Eike U.S. ac- 
creditation systems, the engagement 
of peer reviewers from other schools 
in the state brings multiple perspec- 
tives to the task while stimulating a 


learning process for participants 
that expands their knowledge and 
sharpens their analytical skills. 
Eike research endeavors, the skill- 
ful use of robust quantitative data, 
much of which is comparable 
across schools, with qualitative 
insights developed from looking 
purposefully at teaching and stu- 
dent work and talking to stake- 
holders, allows reviewers to get a 
better understanding of how the 
school is working and what may 
help it improve. 

As an example, the New York 
City Quality Review joins exter- 
nal review to a process of self-as- 
sessment.^® The Quality Review 
focuses on instructional and or- 
ganizational coherence as keys to 
improving student learning. The 
review looks at quality indicators 
related to rigorous, engaging, and 
coherent curricula; effective in- 
struction that yields high-quality 
student work; assessment practices 
that inform instruction; positive, 
inclusive learning environments; a 
supportive culture of learning; use 
of resources to support students’ 
needs; shared goals; teacher sup- 
port and feedback; collaborative 
inquiry; and regular evaluation of 
processes and outcomes.^' School 
self-evaluations precede multi-day 
site reviews by highly trained, ex- 
perienced educators which draw 
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on school-related data; previous 
progress reports; visits to class- 
rooms; observations of teacher 
team meetings; meetings with 
administrators, students, parents, 
and teachers; and examinations 
of curricular artifacts and other 
school-related documents. 

The review results in verbal feed- 
back to the school team and a writ- 
ten report detailing the findings — 
including the school’s strengths 


and areas needing improvement — 
that can guide school improvement 
efforts. 

Massachusetts uses a similar pro- 
cess for reviewing charter schools 
for renewal. Each school prepares 
an accountability plan establishing 
its objectives and measures. The 
Department of Education reviews 
the school’s performance against the 
Charter School Performance Crite- 
ria using this plan and the results of 


a multi-day site visit in which re- 
viewers gather and document evi- 
dence of the school’s performance. 
While onsite, team members con- 
duct focus groups and interviews, 
observe classrooms, and conduct 
a document review guided by an 
evaluation rubric. They prepare a 
report that is publicly posted and 
used to guide decision making 
and ongoing improvement.^^ 


The SQR process has proven an ex- 
tremely effective strategy for help- 
ing schools get an objective look 
at their practices. The process cre- 
ates an evidence base that honors 
the broader goals of education and 
complements test information, 
while providing diagnostics and 
recommendations that are essential 
for any serious improvement ulti- 
mately to occur. When practicing 
educators are among the members 
of the teams, they also learn directly 
about colleagues’ practices and how 
to evaluate education in ways that 
travel back with them to their own 
schools, creating a learning system 
across the state. 

The goal of SQR should be a system 
of review by expert educators and 
peers using robust data and qualita- 
tive inquiry to examine learning in 
individual schools and help them 
determine how to move forward. 


In addition, a capacity to learn from 
these reviews in a more aggregated 
fashion could help build a learning 
system within the state that stimu- 
lates the transfer of knowledge and 
best practices and encourages inno- 
vation, experimentation, evaluation, 
and adaptation. 

The SQR process would be used by 
Figure 4; The School Quality Review 


the CCEE for schools that are iden- 
tified as needing assistance or that 
volunteer for this support. The re- 
view process would be joined with 
an intensive support process in 
which the district and state identify 
and activate the human and other 
resources that are needed to enable 
the school to turn around its prac- 
tices and its students’ performance. 
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The SQR could ultimately become 
available to all schools on a cycli- 
cal basis (typically every fifth year) 
and to schools that volunteer to 
participate more frequently because 
they want the additional help it 
can provide. To facilitate broader 
use, it might be useful to consider 
whether current school accredita- 
tion could be re-conceptualized to 
focus more directly on teaching and 
learning, with leadership from full- 
time trained experts who guide the 
work of the volunteer participants 
on teams that can be more consis- 
tent and effective as a result. 

Assistance to Schools and 
Districts 

The CCEE will need to develop 
a support capacity to work with 
schools or districts that request or 
are identified for improvement assis- 
tance. As this work gets underway, 
there is much to learn from Califor- 
nia’s previous experiences in trying 
to assist low-performing schools. 
These include the creation of SAITs 
under the state’s own Public School 
Accountability Act and the later cre- 
ation of DAITs in response to the 
federal NCEB Eaw. 

Both programs used external con- 
sultants or vendors to oversee in- 
terventions, and both had uneven 
results. Studies of these programs 
demonstrated that frequent barri- 
ers included insufficient engage- 


ment of the vendor with the school 
or district, lack of relational trust 
and stakeholder buy-in, inadequate 
communication, and turnover and/ 
or lack of engagement of district 
leadership. The SAIT process was 
criticized for focusing on a pre-de- 
termined list of interventions that 
providers were expected to monitor 
three times a year. Often, schools 
reported little attention to coaching 
or implementation support.^^ 

The DAIT process sought to attend 
to these concerns with a greater em- 
phasis on implementation, which 
did prove helpful in some circum- 
stances. Still, there were often con- 
cerns raised about inadequate ex- 
pertise on the part of the external 
teams. Districts needed many kinds 
of expertise, ranging from knowl- 
edge about instruction to English 
language learners to fiscal adminis- 
tration and business services to sup- 
port local reform efforts. A better 
fit between the knowledge and skills 
of the DAIT provider and the high- 
priority needs of the district resulted 
in better outcomes in DAIT work. 
Staff in DAIT districts reported 
wanting the state to engage in bet- 
ter screening of the DAIT providers 
and better matching of the service 
provider’s knowledge and abilities 
to the needs of the districts.^'’ 

One successful support structure 
used in North Carolina and Ken- 
tucky is the training and deploy- 


ment of a cadre of Distinguished 
Educators — accomplished teachers, 
principals, and superintendents — 
who are intensively prepared and 
made available to work with schools 
and districts that are engaged in im- 
provement or turnaround efforts. 
These Distinguished Educator 
teams work closely with the districts 
and schools they serve rather than 
appearing periodically to give advice 
or to oversee a list of interventions. 

Another promising strategy is the 
creation of school or district pair- 
ings and networks, connecting 
schools that are struggling to more 
successful schools that face similar 
challenges. This approach to con- 
tinuous improvement has been pio- 
neered in Shanghai, China, and in 
the CORE districts in California 
(see Box 4). As described earlier, 
networking among small groups 
of schools, combined with knowl- 
edge dissemination strategies, has 
been used successfully to support 
improvement in other contexts, in- 
cluding England and Ontario, Can- 
ada. School pairings or networks 
within and across districts could be 
facilitated by COEs and CSAs, or 
by the CCEE. 


SUPPORTING CONTINUOUS IMPROVEMENT IN CALIFORNIA'S EDUCATION SYSTEM ^ 


PACE 


BOX 4; Pairing Schools for Improvement 


The education system in Shang- 
hai has received much attention 
since it emerged at the top of the 
Program in International Stu- 
dent Assessment (PISA) rankings 
in 2009. Shanghais educational 
reforms since the late 1980s have 
focused on building instructional 
capacity and improving the quali- 
ty of schools in the basic education 
system. In 2005, the Shanghai 
government paired urban districts 
with rural districts, so that author- 
ities could collaborate on ways to 
build teachers’ instructional ca- 
pacity and improve school quali- 
ty.^^ The Teachers’ Professional 
Development Institutes affiliated 
with the districts also partici- 
pated by sharing curricula, teach- 
ing materials, and best practices. 
The first round of school pairings 
ended successfully in 2008, and a 
second round of pairings has since 
been implemented. 

Since then, Shanghai has created 
an empowered-management pro- 
gram, in which district officials 
match high-performing and low- 
performing schools in order to 


support improvement. The high- 
performing school is contracted to 
assist the low-performing school. 
Over a period of two years, teachers 
and school leaders from both schools 
move between the two schools shar- 
ing and developing practices. The 
support school creates a detailed 
plan for improving teaching after 
having evaluated the teaching in 
the supported school and described 
what needs to be changed. They 
work together on individual and 
collective professional development 
for teachers and leaders, research 
and lesson study, and developing 
stronger curriculum resources. A 
key goal is to develop a strong lead- 
ership team among a management 
team of administrators and teachers 
who can help develop other teach- 
ers and practices within the school. 
The program has been found to be 
highly successful and is being ex- 
panded: Many schools report im- 
proved student performance and at- 
tainment, as well as improvements 
in other areas. 

In 2013, the California Office to 
Reform Education (CORE) incor- 


porated a school pairing strategy 
into its ESEA waiver application 
as a strategy for transforming low- 
performing schools, core’s ap- 
proach to school improvement 
pairs staffs of demographically 
similar high-performing schools 
with focus and priority schools.^^ 
The schools are matched based 
on specific areas of strengths and 
weaknesses. The school pairings 
may occur within or between dis- 
tricts, and the purpose of the pair- 
ings is to allow for teachers and 
administrators to share ideas and 
intervention practices and to col- 
laboratively design a school im- 
provement plan. Both the high- 
er- and lower-performing schools 
receive training and opportunities 
to learn together and engage in 
the peer review process. The fo- 
cus and priority schools first en- 
gage in a needs assessment, similar 
to a SQR, and use the assessment 
findings to develop a school im- 
provement plan. Then the paired 
schools, called collaborative part- 
ners, work together to support the 
implementation of the school im- 
provement plan. 
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Building the Infrastructure for 
Continuous Improvement 

As we noted above, building and 
sustaining an education system that 
is organized to support continuous 
improvement will ultimately require 
additional action by the state to fa- 
cilitate knowledge organization 
and sharing within the education 
system, to provide support for care- 
ful evaluation of promising poli- 
cies and practices, and to rebuild a 
system of learning supports that 
will ensure that educators have the 
knowledge and skill they need to 
prepare all of California’s students 
for college and careers. Over time, 
the CCEE could expand and evolve 
to take on these tasks, which are es- 
sential supports if its core work is to 
be effective. 

Knowledge Organization and 
Sharing 

The CCEE could support efforts 
to transform struggling schools and 
build professional capacity by help- 
ing to create a statewide learning 
system that explicitly develops and 
disseminates knowledge about best 
practices and successful strategies. 
The new agency can organize and 
share research and best practices 
through its website and dissemina- 
tion activities (e.g., newsletters, con- 
ferences, and school quality review 
activities). It should also document 
and disseminate what is working in 


schools in the state in multiple ways, 
including case studies of and site 
visits to schools that are succeed- 
ing in particular ways (e.g., working 
with new immigrant students; im- 
proving early literacy; or developing 
multi-tiered systems of support for 
students with learning difficulties). 
The CCEE can also develop or lo- 
cate and disseminate tools to sup- 
port local policy and practice, such 
as curriculum resources, educator 
evaluation tools, or student diagnos- 
tic assessments. Einally, it can set up 
and support learning networks that 
allow districts, schools, and educa- 
tors to learn from one another. 

Evaluation Support 

In order to produce and share knowl- 
edge, the CCEE should be aware 
of and able to disseminate evalua- 
tion research about key policies and 
practices. It might eventually spon- 
sor evaluations that are designed 
to add strategically to knowledge 
about how to solve critical problems 
of practice. On the one hand, this 
could include systematic evaluation 
of the CCEE’s own work, which 
could produce an invaluable flow of 
information about what works and 
what doesn’t for different groups 
of students in differently situated 
schools across the state. On the 
other hand, the CCEE might com- 
mission evaluation research on spe- 
cific topics (e.g., classroom use of 
digital technologies, or alternative 


approaches to English language de- 
velopment) where the current state 
of knowledge is relatively weak, to 
be carried out in partnership with 
organizations that specialize in this 
kind of work, including universities 
and research organizations. 

This kind of basic research is essen- 
tial to the long-term effectiveness of 
a continuously improving system, 
which is predicated on the steady 
availability of new and reliable in- 
formation, to take account of the 
new challenges and opportunities 
with which schools and school dis- 
tricts are constantly faced. 

Learning Supports 

Einally, the CCEE could help to re- 
build some of the now-missing in- 
frastructure for professional learning 
in California. Over the last decade, 
the state has lost or greatly reduced 
most of its programs supporting 
professional learning for teachers 
and administrators, including the 
California School Eeadership Acad- 
emy, which trained leaders as well 
as teams for school turnaround, the 
California Subject Matter Projects, 
and the many professional develop- 
ment programs previously attached 
to categorical funding streams that 
are now included in the ECEE 

While the CCEE cannot replace all 
this lost capacity, it will be critically 
important that the state develop 
leadership capacity that is needed 


SUPPORTING CONTINUOUS IMPROVEMENT IN CALIFORNIA'S EDUCATION SYSTEM B| 


PACE 


to support continuously improving 
schools and to transform those that 
are struggling. It would he appro- 
priate and strategic for the CCEE 
to help orchestrate the expertise 
needed to help train mentors, mas- 
ter teachers, coaches, principals, 
and district leaders individually and 
as part of school and district teams 
for the range of challenges they face, 
including new curriculum stan- 
dards, use of assessments to inform 
improvement, school improvement 
initiatives, and turning around 
schools. 

In order to huild local capacity, it 
would also he useful for the CCEE 
to: 

• Orchestrate access to high-quali- 
ty curriculum resources (includ- 
ing instructional materials and 
videotapes of practice) around 
which professional development 
can he organized and on which 
teacher teams can huild, try, and 
refine locally adapted lessons and 
instructional strategies; 

• Provide ongoing training for 
schools to develop effective pro- 
fessional learning communities 
that can analyze student learning 
and school progress in relation to 
practice and engage in ongoing 
improvement; 

• Help districts huild stronger eval- 
uation systems that include effec- 
tive strategies like Peer Assistance 


and Review systems."^” 

Design Principles for CCEE 

The CCEE is an entirely new piece 
of California’s accountahility system, 
and fundamental questions about 
its staffing and operations — as well 
as whether it will have an adequate 
level of funding for the tasks it takes 
on — remain to he answered. As an 
unclaimed and potentially powerful 
resource in California’s policy land- 
scape, the CCEE is already the ob- 
ject of a host of competing views on 
what its core purposes and organiz- 
ing principles should be. 

An appointed board comprising 
five members (a teacher, a local su- 
perintendent, a county superinten- 
dent, and representatives of the SPI 
and the SEE President) will make 
decisions about the design and op- 
eration of the CCEE in the com- 
ing year, with a report to the State 
Board of Education due in October 
2015. Getting these decisions right 
is critical to the CCEE’s success. 

In our view, the central goal of the 
CCEE is to support continuous im- 
provement in the performance of 
California schools and students over 
time, and three fundamental prin- 
ciples should guide decisions about 
organizational design. Eirst, the 
CCEE should employ a core profes- 
sional staff to review intervention 
strategies and oversee technical as- 
sistance activities, while contracting 


with multiple partners to provide 
direct assistance to school districts 
and schools. Second, the CCEE 
should build on existing public in- 
frastructure to the greatest possible 
extent, finally, the design should 
be scalable, to enable the CCEE to 
respond to new expectations and 
growing demand for assistance over 
time with a thoughtful phase-in 
process. 

Professionalism 

Even as many services are delegated 
or contracted out to other agen- 
cies, a core staff (analogous to Her 
Majesty’s Inspectors in the English 
inspectorate system - see Box 3) is 
essential to the effective operation 
of the CCEE. Managing a newly 
created organization that is grow- 
ing rapidly in both the quantity 
and the variety of services that it 
provides will not be easy. Core staff 
should therefore include exemplary 
practitioners to recruit and over- 
see technical assistance providers, 
along with some research support 
to monitor CCEE performance and 
evaluate new resources and practices 
in the state’s education system. In 
an education system that is continu- 
ously improving, the challenges and 
opportunities that schools and dis- 
tricts face will change over time, and 
the CCEE must be staffed in ways 
that enable it to respond effectively 
to new circumstances. 
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Partnership 

To the greatest extent possible, the 
CCEE should build on existing 
infrastructure, supporting and co- 
ordinating the efforts of the many 
agencies that are already engaged 
in the work of improving the per- 
formance of schools and students 
in California. This would clearly 
include work to inform and sup- 
port the technical assistance work 
of COEs and CSAs, and might 
also include collaboration with re- 
searchers in California universities 
to help expand the knowledge base 
for effective intervention in school 
districts and schools. Along with 
a revitalized California State Eead- 
ership Academy, the University of 
California’s Education Evaluation 
Center (now based at UCEA) might 
have an important role to play. The 
CCEE might also be able to bring 
some coherence and consistency to 
the now weakly coordinated efforts 
of the many foundations, non-prof- 
its, universities, and entrepreneurs 
who are working to support contin- 
uous improvement in educational 
performance in California. 

Scalability 

The CCEE should be organized to 
make it readily scalable, along two 
different dimensions. While it must 
be phased in with care, the CCEE 
must also be prepared to expand the 
number of clients that it is able to 


serve. The agency’s hrst responsi- 
bility will be to provide diagnostic 
support and technical assistance 
to schools that volunteer or are re- 
ferred for its services. That means 
that designing an effective School 
Quality Review and developing an 
effective set of assistance strategies 
will be early orders of business. 

In light of the multiple challenges 
facing California’s education system, 
it is virtually certain that the demand 
for technical assistance and support 
from schools and school districts will 
quickly exceed the initial capacity of 
the CCEE to respond. To keep up 
with the demands placed upon it, 
the CCEE must be flexible enough 
to add capacity in a timely way as 
demand for its services increases. 
This could be accomplished in a va- 
riety of ways: through coordination 
with COEs, CSAs, universities, and 
other public agencies that provide 
assistance to schools and school dis- 
tricts; by recruiting a pool of distin- 
guished educators to provide tech- 
nical assistance on a consulting basis 
on behalf of the CCEE; or through 
contracting with multiple technical 
assistance providers including pub- 
lic, non-profit, and for-profit agen- 
cies. 

Soon thereafter, however, the agency 
should consider how to support con- 
tinuous improvement in more sys- 
temic ways, rather than one school 
or district at a time. To do that, it 


will need to develop knowledge pro- 
duction and sharing strategies like 
those identified above. As we have 
noted, California’s new accountabil- 
ity system is itself a work in prog- 
ress, with many important func- 
tions and responsibilities currently 
unassigned or weakly pursued. Eor 
example, many local educators find 
themselves ill equipped to make in- 
formed choices about instructional 
materials or professional develop- 
ment to support the implementa- 
tion of the CCSS. State action will 
be required to fill many of these 
gaps, but at present, no state agency 
has that responsibility. There will 
be pressure on the CCEE to assume 
additional responsibilities, and the 
organization should be structured 
in ways that would enable it address 
some of the key systemic needs that 
will enable districts to build internal 
capacity for continuous improve- 
ment. 

Conclusion 

The implementation of the changes 
that have been launched in Califor- 
nia will be a lengthy process, un- 
folding over the next several years. 
The short-term priorities are suc- 
cessful implementation of Califor- 
nia’s new academic standards and 
the ECEE, but the goal of a continu- 
ously improving education system 
nevertheless remains the central ob- 
jective. Schools and school districts 
are faced with a host of unfamiliar 
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and often difficult implementation 
challenges with little or no guidance 
from the state, and with few obvi- 
ous sources of support or technical 
assistance. The state needs to hold 
up its end of this endeavor, and the 
CCEE is a key agency in providing 
support and technical assistance to 
schools and districts that need ad- 
ditional help. Moving successfully 
from a system driven hy compliance 
to one that is oriented to system 
learning and continuous improve- 
ment will require that policymakers 
develop and pursue a unified long- 
term strategy, grounded in Califor- 
nia’s new accountability system. 
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Appendix A: CORE 
Accountability System 

Perhaps the most comprehensive 
approach to accountahility for con- 
tinuous improvement has been de- 
veloped hy the California Office to 
Reform Education (CORE) districts 
in California, which have huilt on 
California’s multiple measures sys- 
tem under the LCAP and developed 
a multi-dimensional system for in- 
forming school accountahility and 
improvement. These districts (Fres- 
no, Long Beach, Los Angeles, Oak- 
land, San Francisco, Santa Ana, and 
Sanger) joined together and were 
granted a federal flexihility waiver 
under NCLB, which includes the 
accountahility measures shown be- 
low. 

Many of these measures are required 
by the state LCAP, but others, such 
as the non-cognitive skills associated 
with social-emotional learning, are 
locally determined and measured. 


All of these measures are considered 
individually in informing schools 
about their progress and supporting 
ongoing improvement efforts. 

Other indicators used in Califor- 
nia’s LCAP are also reported in 
CORE districts, including measures 
of students’ opportunities to learn 
and parents’ opportunities to be in- 
volved in their children’s education. 
These include: 

• The availability of qualified 
teachers, adequate facilities, and 
necessary materials 

• Student access to a broad curricu- 
lum, including the core subjects 
(including science and technol- 
ogy), the arts, and physical edu- 
cation 

• Student access to college course- 
work and career pathways 

• Evidence of parent participation 
and opportunities for input 

• To meet federal requirements 


for identifying low-performing 
schools, CORE developed a 
School Quality Improvement 
Index comprising weighted mea- 
sures within three domains: 

• Academic (achievement and 
growth, graduation rate, and per- 
sistence rate in Grades 8-10, to- 
gether 60% of the index); 

• Social/Emotional (suspension/ 
expulsion, chronic absenteeism, 
and noncognitive skills, together 
20% of the index); and 

• School/District Culture & Cli- 
mate (stakeholder voice/per- 
ceptions of students, staff, and 
parents; special education iden- 
tification; and English learner 
entry/exit, together 20% of the 
index). 

To couple resource allocations with 
identification ofschool needs, CORE 
directs improvement resources (for- 
mative tasks, student remediation 
courses, professional development 
for teachers) toward any school that 
falls below certain thresholds (e.g., a 
specific pass rate on the tenth-grade 
California High School Exit Exam), 
regardless of the school’s overall 
rankings. CORE has also outlined 
a resource-enriched School Quality 
Improvement process that builds 
professional capacity in schools that 
are identified as priority schools, as 
well as sharing expertise among all 
schools in the consortium.'® 


CORE Accountability Structure 
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Educator, student, and parent perceptions of school climate 
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